The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps.

Credit assignment problem of neural networks refers to evaluating the credit of each network component to the final outputs. For an untrained neural network, approaches to tackling it have made great contributions to parameter update and model revolution during the training phase. This problem on trained neural networks receives rare attention, nevertheless, it plays an increasingly important role in neural network patch, specification and verification. Based on Koopman operator theory, this paper presents an alternative perspective of linear dynamics on dealing with the credit assignment problem for trained neural networks. Regarding a neural network as the composition of sub-dynamics series, we utilize step-delay embedding to capture snapshots of each component, characterizing the established mapping as exactly as possible. To circumvent the dimension-difference problem encountered during the embedding, a composition and decomposition of an auxiliary linear layer, termed minimal linear dimension alignment, is carefully designed with rigorous formal guarantee. Afterwards, each component is approximated by a Koopman operator and we derive the Jacobian matrix and its corresponding determinant, similar to backward propagation. Then, we can define a metric with algebraic interpretability for the credit assignment of each network component. Moreover, experiments conducted on typical neural networks demonstrate the effectiveness of the proposed method.

Open-vocabulary scene understanding aims to localize and recognize unseen categories beyond the annotated label space. The recent breakthrough of 2D open-vocabulary perception is largely driven by Internet-scale paired image-text data with rich vocabulary concepts. However, this success cannot be directly transferred to 3D scenarios due to the inaccessibility of large-scale 3D-text pairs. To this end, we propose to distill knowledge encoded in pre-trained vision-language (VL) foundation models through captioning multi-view images from 3D, which allows explicitly associating 3D and semantic-rich captions. Further, to facilitate coarse-to-fine visual-semantic representation learning from captions, we design hierarchical 3D-caption pairs, leveraging geometric constraints between 3D scenes and multi-view images. Finally, by employing contrastive learning, the model learns language-aware embeddings that connect 3D and text for open-vocabulary tasks. Our method not only remarkably outperforms baseline methods by 25.8% $\sim$ 44.7% hIoU and 14.5% $\sim$ 50.4% hAP$_{50}$ on open-vocabulary semantic and instance segmentation, but also shows robust transferability on challenging zero-shot domain transfer tasks. Code will be available at https://github.com/CVMI-Lab/PLA.

由于其在多语言翻译，自动驾驶等方面的广泛应用，因此场景文本识别引起了近年来的兴趣。在本报告中，我们描述了我们对词汇表场上的解决方案的解决方案，该解决方案是词汇表场上的文本理解（OOV-ST）挑战，旨在从自然场景图像中提取胶卷外（OOV）单词。我们基于OCLIP的模型在H-Mean中获得28.59 \％，在ECCV2022 TIE Workshop中对OOV挑战的端到端OOV单词识别曲目排名第一。

该报告介绍了我们对ECCV 2022挑战的获胜者解决方案，挑战了播放视频的文本理解（OOV-ST）：裁剪单词识别。这项挑战是在所有内容（TIE）中的ECCV 2022讲习班的背景下举行的，该研讨会（TIE）旨在从自然场景图像中提取出播出的单词。在竞争中，我们首先在合成数据集上进行预训练，然后在训练集中对模型进行数据增强进行微调。同时，针对长期和垂直文本进行了专门训练的另外两个型号。最后，我们将不同模型的输出与不同的层，不同的骨干和不同种子结合在一起。当考虑使用唱歌内和播放量的单词时，我们的解决方案的总体单词准确性为69.73％。

大多数现有场景文本检测器都集中于检测字符或单词，这些字符或单词仅由于缺少上下文信息而捕获部分文本消息。为了更好地理解场景中的文本，更需要检测上下文文本块（CTB），该文本块由一个或多个积分文本单元（例如，字符，单词或短语）组成，自然阅读顺序并传输某些完整的文本消息。本文介绍了上下文文本检测，这是一种检测CTB的新设置，以更好地理解场景中的文本。我们通过双重检测任务制定新设置，该任务首先检测积分文本单元，然后将其分组为CTB。为此，我们设计了一种新颖的场景文本群集技术，将整体文本单元视为令牌，并将它们（属于同一CTB）分组为有序的令牌序列。此外，我们创建了两个数据集Scut-ctw-context和rects-context，以促进未来的研究，其中每个CTB都由有序的积分文本单元很好地注释。此外，我们介绍了三个指标，这些指标以局部准确性，连续性和全球准确性来衡量上下文文本检测。广泛的实验表明，我们的方法准确地检测到CTB，这些CTB有效地促进了下游任务，例如文本分类和翻译。该项目可在https://sg-vilab.github.io/publication/xue20222contextual/上获得。

最近，视觉预训练（VLP）技术通过共同学习视觉和文本表示，从而极大地使各种视力语言任务受益匪浅，这是由于场景中文本中丰富的视觉和文本信息而直觉上有助于光学角色识别（OCR）任务图片。但是，这些方法无法很好地应对OCR任务，因为实例级文本编码和图像文本对采集的难度（即其中的图像和捕获的文本）。本文提出了一种弱监督的预训练方法OCLIP，可以通过共同学习和对齐视觉和文本信息来获取有效的场景文本表示。我们的网络由一个图像编码器和角色吸引的文本编码器组成，该文本编码器分别提取视觉和文本特征，以及一个视觉文本解码器，该解码器模拟了文本和视觉特征之间的相互作用，以学习有效的场景文本表示。通过学习文本功能，预先训练的模型可以通过角色意识很好地参加图像中的文本。此外，这些设计可以从弱注释的文本（即图像中的部分文本中没有文本边界框中的部分文本）进行学习，从而极大地减轻数据注释约束。 ICDAR2019-LSVT中弱注释图像的实验表明，我们的预训练模型分别将其权重转移到其他文本检测和发现网络时，将F-评分提高+2.5 \％和+4.8 \％。此外，所提出的方法在多个公共数据集（例如，总文本和CTW1500的+3.2 \％和+1.3 \％）上始终超过现有的预训练技术。

深度学习（DL）模型为各种医学成像基准挑战提供了最先进的性能，包括脑肿瘤细分（BRATS）挑战。然而，局灶性病理多隔室分割（例如，肿瘤和病变子区）的任务特别具有挑战性，并且潜在的错误阻碍DL模型转化为临床工作流程。量化不确定形式的DL模型预测的可靠性，可以实现最不确定的地区的临床审查，从而建立信任并铺平临床翻译。最近，已经引入了许多不确定性估计方法，用于DL医学图像分割任务。开发指标评估和比较不确定性措施的表现将有助于最终用户制定更明智的决策。在本研究中，我们探索并评估在Brats 2019-2020任务期间开发的公制，以对不确定量化量化（Qu-Brats），并旨在评估和排列脑肿瘤多隔室分割的不确定性估计。该公制（1）奖励不确定性估计，对正确断言产生高置信度，以及在不正确的断言处分配低置信水平的估计数，（2）惩罚导致更高百分比的无关正确断言百分比的不确定性措施。我们进一步基准测试由14个独立参与的Qu-Brats 2020的分割不确定性，所有这些都参与了主要的Brats细分任务。总体而言，我们的研究结果证实了不确定性估计提供了分割算法的重要性和互补价值，因此突出了医学图像分析中不确定性量化的需求。我们的评估代码在HTTPS://github.com/ragmeh11/qu-brats公开提供。

高光谱图像（HSI）分类一直是决定的热门话题，因为高光谱图像具有丰富的空间和光谱信息，并为区分不同的土地覆盖物体提供了有力的基础。从深度学习技术的发展中受益，基于深度学习的HSI分类方法已实现了有希望的表现。最近，已经提出了一些用于HSI分类的神经架构搜索（NAS）算法，这将HSI分类的准确性进一步提高到了新的水平。在本文中，NAS和变压器首次合并用于处理HSI分类任务。与以前的工作相比，提出的方法有两个主要差异。首先，我们重新访问了先前的HSI分类NAS方法中设计的搜索空间，并提出了一个新型的混合搜索空间，该搜索空间由空间主导的细胞和频谱主导的单元组成。与以前的工作中提出的搜索空间相比，所提出的混合搜索空间与HSI数据的特征更加一致，即HSIS具有相对较低的空间分辨率和非常高的光谱分辨率。其次，为了进一步提高分类准确性，我们尝试将新兴变压器模块移植到自动设计的卷积神经网络（CNN）上，以将全局信息添加到CNN学到的局部区域的特征中。三个公共HSI数据集的实验结果表明，所提出的方法的性能要比比较方法更好，包括手动设计的网络和基于NAS的HSI分类方法。特别是在最近被捕获的休斯顿大学数据集中，总体准确性提高了近6个百分点。代码可在以下网址获得：https：//github.com/cecilia-xue/hyt-nas。

Leveraging the advances of natural language processing, most recent scene text recognizers adopt an encoder-decoder architecture where text images are first converted to representative features and then a sequence of characters via `sequential decoding'. However, scene text images suffer from rich noises of different sources such as complex background and geometric distortions which often confuse the decoder and lead to incorrect alignment of visual features at noisy decoding time steps. This paper presents I2C2W, a novel scene text recognition technique that is tolerant to geometric and photometric degradation by decomposing scene text recognition into two inter-connected tasks. The first task focuses on image-to-character (I2C) mapping which detects a set of character candidates from images based on different alignments of visual features in an non-sequential way. The second task tackles character-to-word (C2W) mapping which recognizes scene text by decoding words from the detected character candidates. The direct learning from character semantics (instead of noisy image features) corrects falsely detected character candidates effectively which improves the final text recognition accuracy greatly. Extensive experiments over nine public datasets show that the proposed I2C2W outperforms the state-of-the-art by large margins for challenging scene text datasets with various curvature and perspective distortions. It also achieves very competitive recognition performance over multiple normal scene text datasets.